<!--
  ~ Copyright (C) 2011 Brockmann Consult GmbH (info@brockmann-consult.de)
  ~
  ~ This program is free software; you can redistribute it and/or modify it
  ~ under the terms of the GNU General Public License as published by the Free
  ~ Software Foundation; either version 3 of the License, or (at your option)
  ~ any later version.
  ~ This program is distributed in the hope that it will be useful, but WITHOUT
  ~ ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or
  ~ FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for
  ~ more details.
  ~
  ~ You should have received a copy of the GNU General Public License along
  ~ with this program; if not, see http://www.gnu.org/licenses/
  -->

<html>
<head>
    <title>Pixel Extraction</title>
    <link rel="stylesheet" href="../style.css">
</head>

<body>

<table class="header">
    <tr class="header">
        <td class="header">&nbsp;
            Pixel Extraction
        </td>
        <td class="header" align="right"><a href="nbdocs://org.esa.snap.snap.help/org/esa/snap/snap/help/docs/general/overview/SnapOverview.html"><img
                src="../images/snap_header.jpg"
                border=0></a>
        </td>
    </tr>
</table>

<h3>Introduction</h3>

<p>
    Basically, the Pixel Extraction Tool allows to extract the pixel values given by a user-specified list of
    geographic coordinates from a given list of data products. However, a number of additional parameters may be passed
    to the tool, for example the size of the area around the geographic coordinate which is to be considered.
</p>

<h3>The Output Format</h3>

<p>
    The Pixel Extraction Tool generates output in the widely used and very general CSV-format, using tabstops as
    separators.<br/>
    It writes a file for each product type it has found within its input products (such as MER_FRS), each starting with
    a header looking like this:<br/><br/>
    <code>
        # SNAP pixel extraction export table<br/>
        #<br/>
        # Window size: 1<br/>
        # Expression: l1_flags.LAND_OCEAN<br/>
        # Created on: 2010-09-17 14:53:21<br/>
    </code><br/>
    The header is followed by the table with the actual data; it might look like the following:<br/><br/>
<table>
    <tbody>
    <tr>
        <td>Expression result</td>
        <td>ProdID</td>
        <td>CoordID</td>
        <td>Name</td>
        <td>Latitude</td>
        <td>Longitude</td>
        <td>PixelX</td>
        <td>PixelY</td>
        <td>Date(yyyy-MM-dd)</td>
        <td>Time(HH:mm:ss)</td>
        <td>radiance_1</td>
        <td>radiance_2</td>
    </tr>
    <tr>
        <td>true</td>
        <td>0</td>
        <td>1</td>
        <td>pin_1</td>
        <td>41.31899</td>
        <td>2.1386652</td>
        <td>459.0</td>
        <td>853.0</td>
        <td>2003-08-09</td>
        <td>12:14:16</td>
        <td>157.6424</td>
        <td>163.6796</td>
    </tr>
    <tr>
        <td>true</td>
        <td>1</td>
        <td>1</td>
        <td>pin_1</td>
        <td>41.31899</td>
        <td>2.1386652</td>
        <td>459.0</td>
        <td>853.0</td>
        <td>2005-08-09</td>
        <td>18:15:26</td>
        <td>234.534</td>
        <td>143.423</td>
    </tr>
    <tr>
        <td>false</td>
        <td>3</td>
        <td>2</td>
        <td>pin_2</td>
        <td>43.3452</td>
        <td>12.1232</td>
        <td>539.0</td>
        <td>533.0</td>
        <td>2001-07-05</td>
        <td>12:07:86</td>
        <td>423.522194</td>
        <td>156.625496</td>
    </tr>
    </tbody>
</table>

<p>
    In the following table, the entries are explained in detail.
</p>

<table>
    <thead>
    <tr>
        <th>Field</th>
        <th>Description</th>
    </tr>
    </thead>
    <tbody>
    <tr>
        <td>Expression result</td>
        <td>True if the value matches the expression, false otherwise. Is omitted when expression is used for
            filtering.
        </td>
    </tr>
    <tr>
        <td>ProdID</td>
        <td>An ID for the product with relevance within the extracted data.</td>
    </tr>
    <tr>
        <td>CoordID</td>
        <td>An ID for the coordinate with relevance within the extracted data.</td>
    </tr>
    <tr>
        <td>Name</td>
        <td>The coordinate's name, if available.</td>
    </tr>
    <tr>
        <td>Latitude</td>
        <td>The coordinate's latitude value.</td>
    </tr>
    <tr>
        <td>Longitude</td>
        <td>The coordinate's longitude value.</td>
    </tr>
    <tr>
        <td>PixelX</td>
        <td>The data product's pixel value in x direction corresponding to the latitude value.</td>
    </tr>
    <tr>
        <td>PixelY</td>
        <td>The data product's pixel value in y direction corresponding to the longitude value.</td>
    </tr>
    <tr>
        <td>Date(yyyy-MM-dd)</td>
        <td>The data product's sensing date.</td>
    </tr>
    <tr>
        <td>Time(HH:mm:ss)</td>
        <td>The data product's sensing time.</td>
    </tr>
    <tr>
        <td>radiance_1</td>
        <td>Exemplary: the first of the bands considered for value extracting.</td>
    </tr>
    <tr>
        <td>radiance_2</td>
        <td>Exemplary: the second of the bands considered for value extracting.</td>
    </tr>
    </tbody>
</table>


<h3>The User Interface</h3>

<p>
    The user interface of the Pixel Extraction Tool consists of two panes. The first is an Input/Output-pane, which
    allows the user to specify
<ul>
    <li>the input paths of the products that shall be considered</li>
    <li>an output directory for the extracted values</li>
    <li>a prefix the output files shall share</li>
    <li>a field that allows the user to extract the time from the product filename. This field may come in handy when
        there are products that don't carry time information readable by the product reader.
    </li>
</ul>
The Input/Output-pane is explained in detail within the next section.
The second pane is dedicated to specify a number of parameters and is explained thoroughly in the section after the
next.
</p>

<h4>The Input/Output Pane</h4>

<p>
    The Input/Output Pane allows to specify the input products and the output mode.
</p>

<p align="center">
    <img src="images/ioform.png"/><br/>
    <span style="font-size: x-small; ">The Input/Output Pane</span>
</p>

<p class="i1">
    <b>Input paths:</b>
    The list contains the data products already chosen for extraction as well as paths to be searched
    for data products. Using the <img src="images/plus.png" alt="plusbutton">-Button, data products can be added to
    the list in multiple ways:
<ul>
    <li><b style="color:#3333cc">Add product(s)...</b> by choosing a product already opened in the Sentinel Toolbox. In front of the
        products name you can see the index of the product e.g. [1].<br><i>
            <small>See line 1 of "Source paths:" field in the Input/Output Pane above.</small>
        </i></li>
    <li><b style="color:#3333cc">Add product file(s)...</b> by choosing one or more products from the file
        system.<br><i>
            <small>See line 2 of "Source paths:" field in the Input/Output Pane above.</small>
        </i></li>
    <li><b style="color:#3333cc">Add directory(s)...</b> by specifying one or more directories and defining a search
        pattern e.g. "*.nc* for netcdf products.<br><i>
            <small>See line 3 of "Source paths:" field in the Input/Output Pane above.</small>
        </i></li>
    <li><b style="color:#3333cc">Add directory recursively...</b> or by specifying a directory recursively (indicated by
        '**' in the list) and defining a search pattern e.g. "*.dim* for DIMAP products.<br><i>
            <small>See line 4 of "Source paths:" field in the Input/Output Pane above.</small>
        </i></li>
</ul>

<p class="i1">
    <b>Time extraction:</b>
    If the products carry no time information that is readable by the reader, it is possible to extract the time
    information from the product filename. In order to do so, the <b>date/time pattern</b> needs to be provided.
    This date pattern can be composed of the following components:
<ul>
    <li>year, given as <i><b>yyyy</b></i></li>
    <li>month, given as <i><b>MM</b></i></li>
    <li>days in month, given as <i><b>dd</b></i></li>
    <li>day in year, given as <i><b>DDD</b></i></li>
    <li>hour of day, in 12 hour format, given as <i><b>hh</b></i></li>
    <li>hour of day, in 24 hour format, given as <i><b>HH</b></i></li>
    <li>minute of hour, given as <i><b>mm</b></i></li>
    <li>second of minute, given as <i><b>ss</b></i></li>
</ul>
<p class="i1">
    , which may appear in arbitrary order. Following some examples which for date format to parse by pattern:
<ul>
    <li><code>yyyyMMdd</code> would parse a string like <code>20111103</code>
        and leads to a date <code>2011-11-03 00:00:00 UTC</code></li>
    <li><code>yyyyMMdd_hh_mm_ss</code> would parse a string like <code>20061126_08_56_12</code>
        and leads to a date <code>2006-11-26 08:56:12 UTC</code></li>
    <li><code>yyyyMMdd-HH:mm:ss</code> would parse a string like <code>20170610-12:36:46</code>
        and leads to a date <code>2017-06-10 12:36:46 UTC</code></li>
    <li><code>yyyyDDD</code> would parse a string like <code>2010159</code>
        and leads to a date <code>2010-06-08 00:00:00 UTC</code></li>
</ul>
<p class="i1">
    The <b>pattern in the filename</b> needs to be provided, in order to tell the software where the pattern occurs within the filename and
    if it is the start or end date.
    The pattern must contain at least one of the placeholders <code>${startDate}</code> and <code>${endDate}</code>. Examples:
    <ul>
        <li><code>sst_*_${startDate}.nc</code></li>
        <li><code>sst_*_${startDate}_${endDate}*.nc</code></li>
        <li><code>LC8......${startDate}*.nc</code></li>
    </ul>
<p class="i1">
    The '<code>*</code>' is a placeholder for multiple characters while the '<code>.</code>' is a placeholder for a single character.
</p>

<p class="i1">
    <b>Output directory:</b>
    All output is written to the selected directory.
</p>

<p class="i1">
    <b>File prefix:</b>
    The specified file prefix is used to prefix the files which are written.
</p>

<h4>The Parameters Pane</h4>

<p>
    The Parameters Pane allows to specify several parameters used for the extraction.
</p>

<p align="center">
    <img src="images/parametersform.png"/><br/>
    <span style="font-size: x-small; ">The Parameters Pane</span>
</p>

<p class="i1">
    <b>Coordinates:</b>
    The coordinates table shows the user-specified geographic coordinates which are used to extract the values.
    They consist of a name and their latitude/longitude-values. Initially, the table is filled with the pins of the
    product currently selected in the Sentinel Toolbox. Additional coordinates may be added by
<ul>
    <li>creating new coordinates</li>
    <li>reading them from a pin file.
        In addition to the standard format of a pin file, the coordinates file can have a 'DateTime' column.
        The format of the time value follows the ISO8601 pattern: yyyy-MM-dd'T'HH:mm:ss. UTF-8 encoding is preferred.<br>
        <b>Example:</b>
        <pre>
Name	Lon	Lat	DateTime
P1	29.8410	31.1748	2009-06-07T08:27:11
P2	31.2213	31.5987	2009-06-07T08:27:04
P3	32.3835	31.1532	2009-06-07T08:27:03
        </pre>
    </li>
    <li>reading them from a file in the output CSV-format described above.
        This allows to compare the results from multiple runs of the Pixel Extraction Tool.
    </li>
</ul>

</p>

<p class="i1">
    <b>Allowed time difference:</b>
    Specifies how big the difference might be between the time of a pixel and the time of the coordinate.
</p>

<p class="i1">
    <b>Export:</b>
    Specifies if bands, tie-point grids and/or masks should be included in the output.
</p>

<p class="i1">
    <b>Window size:</b>
    The window size specifies the number of pixels surrounding the pixels derived from the given geographic
    coordinates to be exported. That is, when setting the window size to 3, the values for 3x3=9 pixels will be
    extracted.
    The window size must be odd.
</p>

<p class="i1">
    <b>Pixel value aggregation method:</b>
    The user can choose between a number of aggregation methods when a window size bigger than 1x1 is used. Choosing
    such an aggregation method, the pixel values from the window are all aggregated, so that only a single aggregated
    value for each coordinate will be exported. If <b>no aggregation</b> is selected, values of all pixels in the
    window will be exported. The aggregation methods that may be used are:
<ul>
    <li><b>min</b>, exports the minimum of the window values</li>
    <li><b>max</b>, exports the maximum of the window values</li>
    <li><b>mean</b>, exports mean and standard deviation of the window values</li>
    <li><b>median</b>, exports the median of the window values</li>
</ul>
</p>

<p class="i1">
    <b>Expression:</b>
    A band maths expression can be specified to filter the values to output. The "Edit Expression..."-button
    opens the expression editor, which can be used for easily editing an expression. It is only available if the
    input product list of the "Input/Output"-pane contains at least one data product; however, in any case an expression
    may manually be declared.<br>
    There are two options of how to use the expression: either the expression is used to filter the values to output,
    that is, values which do not satisfy the expression are ignored. In the other case, the information of the value met
    the expression is added to the output.
    The specified aggregation will be performed if at least one pixel is marked as valid by this expression within the aggregation window.
</p>

<p class="i1">
    <b>Sub-scenes:</b>
    Enables the export of sub-scenes. The scenes contain at least all coordinates (including the specified window)
    found in one product. Additionally the size of a border can be specified by which the exported scene size extended.
</p>

<p class="i1">
    <b>Google Earth output:</b>
    All output coordinates are collected in one KMZ file, which can be used to display the points in Google Earth.
</p>

<p class="i1">
    <b>Match with original input:</b>
    The pixel extraction allows outputting of correlative data together with the data that is extracted. This checkbox
    is only enabled if coordinates that carry correlative data have been added.
</p>

<p>&nbsp;</p>
<hr/>

<p><i>Copyright &copy; 2010 by Brockmann Consult (beam-issues@brockmann-consult.de). All rights reserved.</i></p>

</body>
</html>
